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Abstract 

Pi I We consider a stochastic system whose uncontrolled state dynamics are modelled 

by a general one-dimensional Ito diffusion. The control effort that can be applied to 
this system takes the form that is associated with the so-called monotone follower 
problem of singular stochastic control. The control problem that we address aims 
at maximising a performance criterion that rewards high values of the utility derived 
from the system's controlled state but penalises any expenditure of control effort. This 
problem has been motivated by applications such as the so-called goodwill problem in 

ff^ I which the system's state is used to represent the image that a product has in a market, 

■^ ■ while control expenditure is associated with raising the product's image, e.g., through 

advertising. We obtain the solution to the optimisation problem that we consider in a 
closed analytic form under rather general assumptions. Also, our analysis establishes a 
number of results that are concerned with analytic as well as probabilistic expressions 

C^ . for the first derivative of the solution to a second order linear non-homogeneous ordi- 

nary differential equation. These results have independent interest and can potentially 
be of use to the solution of other one-dimensional stochastic control problems. 
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1 Introduction 

We consider a stochastic system whose state is modelled by the controlled, one-dimensional, 
positive Ito diffusion 

dXt = b{Xt) dt + dZt + a{Xt) dWt, Xq = x > 0, 

where W is a. standard one-dimensional Brownian motion and the controlled process Z is 
a caglad increasing process. The objective of the optimisation problem is to maximise the 
performance criterion 

J^(Z) =limsupE / e-^'t'^'h{Xt)dt- I e-^'^'^' k{Xt) o dZt , (1) 

T^oo \_Jo 



over all admissible choices of Z, where 
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= [ r{X^)dv, 









and 

/ e-^^ k{Xt) o dZt = e-^^ k{Xt)dZ^+ J2 / e-^^ k{Xt + s)ds. (2) 

Jo Jo 0<t<T -^^ 

This stochastic control problem has been motivated by the following application. Consider 
a company marketing a given product. This product has an image in a given market where 
it is being sold that evolves randomly over time. We use the random variable Xt to model 
the product's image at time t, for t > 0. The company marketing the product can raise its 
image by means of costly interventions such as advertising. We model the effect of these 
actions by means of the controlled process Z. The company's objective is to maximise the 
expected discounted utility derived from the product's image minus the expected discounted 
"dis-utility" resulting from intervention costs, which is reflected in the structure of the per- 
formance criterion given by ([T]). 

Optimal control problems motivated by applications such as the one discussed briefly 
above have a long history and can be traced back to Nerlove and Arrow [NA62] . who use 
deterministic dynamics to model the evolution of the product's image and consider a multi- 
objective performance criterion. Since then, deterministic optimal control models addressing 
this type of applications have attracted significant interest (see Buratto and Viscolani [BV02J 
and references therein). Several models in which the product's image evolves randomly over 
time, which are more realistic, and result in stochastic optimisation problems have also been 
studied in the literature (see Marinelli |Mar06] and references therein). 

Singular stochastic control was introduced by Bather and Chernoff |BC67] who consid- 
ered a simplified model of spaceship control. In their seminal paper, Benes, Shepp and 



Witsenhausen |BSW80] were the first to solve rigorously an example of a finite-fuel singular 
control problem. Since then, the area has attracted considerable interest in the literature. 
Alvarez [AMRoT] . Chow, Menaldi and Robin |CMR85j . Davis and Zervos |DZ98j . Fleming 
and Soner |FS93i Chapter VIII], Harrison and Taksar [ HT83J . Jacka |J83[ IJ02J . Karatzas 
P^a83] . K0bila jK^. Ma [M92] . 0ksendal [001], Shreve, Lehoczky and Cavers |SLC84] . 
Soner and Shreve [SS89J, Sun |Su87] . Zhu |Z92j . provide an incomplete list, in alphabetical 
order, of further important contributions. Other related contributions include Menaldi and 
Robin |MR84j . Weerasinghe |W02j . and Jack and Zervos |JZ06j who solve singular control 
problems with long-term average rather than discounted criteria. With regard to the struc- 
ture of the performance criterion that we consider, penalising the expenditure of control 
effort by means of integrals as in ([2]) was introduced by Zhu |Z92] and was later adopted by 
Davis and Zervos |DZ98j and Jack and Zervos |JZ06j . 

We solve the problem that we consider by constructing a solution to the associated 
Hamilton- Jacobi-Bellman (HJB) equation in closed analytic form, under general assump- 
tions. This is possible in the generality that we consider because the control problem's state 
space is one-dimensional. Explicitly solvable control problems have attracted significant in- 
terest in the literature for several reasons. First, some of them, such as the one that we solve 
here, are motivated by real-life applications. Second, they reveal the qualitative nature of the 
associated optimal control tactics, and they provide special cases that can be used to assess 
the efficiency of numerical techniques devised to address more complex problems, which is a 
major issue. The majority of such control problems assume that the system's uncontrolled 
dynamics are modelled by a Brownian motion with drift or a geometric Brownian motion. To 
the best of our knowledge, Alvarez |A01] and Jack and Zervos |JZ06] are the only references 
in the singular stochastic control literature in which closed-form solutions are derived when 
the system's uncontrolled dynamics are modelled by a general one-dimensional Ito diffusion. 
The latter reference considers a long-term average rather than an expected discounted per- 
formance criterion such as the one given by ([1]). On the other hand, Alvarez |A01] assumes 
that h = and that A; < is constant. The introduction of a non-trivial running payoff 
function h and a non-trivial running control expenditure function k of sign opposite to the 
one of h gives rise to a genuinely new problem that involves new analysis. At this point, 
we should mention that we are not aware of any reference addressing the problem that we 
solve, even for simple stochastic dynamics. 

The paper is organised as follows. Section [2] is concerned with the formulation of the 
singular stochastic control problem that we solve. In this section, we also develop all of 
the assumptions on the problem data that we make in the paper, and we prove that these 
are sufficient for our optimisation problem to be well-posed. Section [3] is concerned with 
properties of the solution to a non- homogeneous second-order linear ordinary differential 
equation that plays an important role in our analysis. All of the claims that we make there 
without proof are standard and can be found in several references, including Feller |F54] . 
Breiman [B68] . Mandl |Man68j . Ito and McKean [1M96] . Karlin and Taylor |KT81j . Rogers 
and Williams |RW00j . and Borodin and Salminen )BS02j . The results that we prove are new 



and have independent interest because they can be of use in the solution of other stochastic 
control problems. We should note that these results can be generalised with little effort to 
account for the case where the underlying diffusion can take values in an arbitrary interval 
X e R rather than in ]0,cxd[. Also, some of the assumptions made here can be relaxed. 
However, we decided against such relaxations because they would comphcate significantly the 
exposition. In Section HJ we solve the stochastic control problem that we consider. Finally, 
Section [5] is concerned with special cases that arise when h is a power utility function, k 
and r are constants, and the uncontrolled state space dynamics are modelled by a geometric 
Brownian motion (Section 15.11) or a mean-reverting square-root process such as the one in 
the Cox-Ingersoll-Ross interest rate model (Section 15. 2p . 

2 The singular stochastic control problem 

We fix a filtered probability space (fi, JF, JF^, P) satisfying the usual conditions and carrying 
a standard one-dimensional (jFj)-Brownian motion W. We consider a stochastic system 
whose uncontrolled dynamics are modelled by the Ito diffusion associated with the stochastic 
differential equation (SDE) 

rfX° = b{Xf) dt + a{X^) dWt, X^ = x> 0, (3) 

and we make the following assumption. 

Assumption 1 The functions 6, cr : ]0, oo[ ^ M are C^, a' is locally Lipschitz, and cr^(x) > 0, 
for all X > 0. O 

This assumption implies that (^ has a unique strong solution. It also implies that the scale 
function pxo and the speed measure mxo given by 



/ f'-^ bis) \ 
Pxo (c) = 0, p'xo (x) = exp ( -2 / ^^--y ds j 



(4) 



and 



2 
mxo{dx) = dx, (5) 

respectively, for some c > fixed, are well-defined. Additionally, we assume that the solution 
to ([3]) is non-explosive, so that, given any initial condition x, X° g]0, oo[, for all t > 0, with 
probability 1. 

Assumption 2 The Ito diffusion X^ defined by ([3]) is non-explosive. D 



Feller's test for explosions (see Theorem 5.5.29 in Karatzas and Shreve |KS88] ) provides a 
necessary and sufficient condition for this assumption to hold true. Indeed, if we define 



lxo{x) 



[pxo{x) -pxo{s)] mxo{ds) 



then Assumption [2] is satisfied if and only if lim^^o ^xo(^) = lim^^oo ^xo{x) = cxd. 
Now, we model the system's controlled dynamics by the SDE 

dXt = b{Xt) dt + dZt + a{Xt) dWt, Xq = x > 0, 



(6) 



where the controlled process Z is an increasing process. With each admissible intervention 
strategy Z (see Definition [T] below), we associate the performance criterion 



Jx{Z) = limsupE 



where 



a; 



-a; 



r(X) 



r{X) 



h{Xt) dt 



-K 



r(X) 



r(X„) dv, 



for some functions h :]0, cx3[— ^ M and fc, r :]0, cx3[- 
defined by 



k{Xt)odZt , (7) 



(8) 



The integral with respect to Z is 



>'-(^), 



k{Xt)odZt= / e-^* k{X^)dZ',+ Y. 



0<t<T 



AZt 



-,-A 



r(X) 



k{Xt + s) ds, (9) 



where Z'^ is the continuous part of the increasing process Z. 



Definition 1 The family A of all admissible intervention strategies is the set of all (J^t)- 
adapted caglad processes Z with increasing sample paths such that Zq = 0, ([6]) has a unique 
non-explosive strong solution, and 



E 



e-^*''''A;(X,)odZi 



Uo 



< cx), for all T > 0. 



The objective of our control problem is to maximise Jx over all admissible strategies. Ac- 
cordingly, we define the problem's value function v by 

v{x) = sup Jx{Z), for X > 0. 
zeA 

For our optimisation problem to be well-posed and to admit a solution that conforms 
with economic intuition, we need to make additional assumptions. 



Assumption 3 The discounting factor r is C^, r' is locally Lipschitz, and there exists a 
constant r^ > such that r{x) > vq, for all a; > 0. D 

Our analysis will also involve the SDE 

dY^^ = /i(y;°) dt + a(y;°) dWt, y^ = x> o, (lo) 

where 

fx{x) = b{x) + a{x)a'{x) - y{x)^. (11) 

In the presence of Assumptions [H and [31 this SDE has a unique strong solution. Also, a 
short calculation shows that the scale function pyo of this diffusion satisfies 

Pyo{x) := exp - / -^f^ds = -—- — ^--p^oix), 12 

V Jc ^ns) J r{c) a^{x) 

where pxo is the scale function of the diffusion A° defined by (j4]). We make the following 
additional assumption. 

Assumption 4 The Ito diffusion Y^ defined by fITOl) is non-explosive. D 

Remark 1 It would be of interest to establish conditions under which Assumption [2] implies 
Assumption |H We have not been able to address this issue in a comprehensive manner. 
However, we note that sufficient conditions can be established by appealing to appropriate 
comparison theorems for solutions to SDEs. To illustrate this claim, suppose that 

a{x)a'{x)-^cx^{xf-fl>0, for all x > 0, (13) 

r[x) 

and that 

fi is Lipschitz continuous in [1, cxd[. (14) 

In this case. Assumption [2], flT^ and the comparison theorem for solutions to SDE's in 
Karatzas and Shreve |KS88l Proposition 5.2.18] imply that y° does not explode at 0. Fur- 
thermore, (IT^ implies that Y^ does not explode at oo. To see this claim, we argue by contra- 
diction, and we assume that there exists an (jF4)-stopping time t^o such that P(roo < C)o) > 
and hm^^T-oo ^° = ^^- Since y° does not explode at 0, there exist e, C > such that 

p ( Too < OO, inf y;° > £ ) > C- (15) 

\ t<Tac I 



Given such constants, let yUg, a^ : M ^ M be any Lipschitz continuous functions such that 
/ie(x) = fi{x) and <Js{x) = <y{x), for all x > e, let V^ be the solution to 

dr/ = fie (r/) dt + ae (F/) dWu Y^ = x> 0, 

and let r^ be the first hitting time of ]0,£:] by V^ . The Lipschitz continuity of /Xg and a^ 
imply that V/ is real- valued, for all t > 0. However, this observation provides the required 
contradiction because Y^ = y/, for all t < r^, so (TT5l) cannot be true. D 

Throughout the paper, we denote by K the function defined by 



K(x) 



k{s) ds, 



and, given a C^ function w, we denote by Cxw the function given by 



Cxw{x) 



1„2 



a {x)w"{x) +b{x)w'{x) — r{x)w{x). 



Also, we define 

Q(x) =h{x) + CxK{x). 
We can now complete the list of assumptions made in the paper. 



(16) 

(17) 
(18) 



Assumption 5 The following conditions hold: 

(a) The running payoff function h is C^ and the function h/r is bounded from below. 

(b) The running cost function k is C^. Also, k{x) > 0, for all x > 0, and the function K 
defined by flT6|) is real-valued. 

(c) The problem data is such that 



p{x) :-- 



r^(x) — r{x)b'{x) + r'{x)b{x) 



rix) 



> tq, for all a; > 0, 



(19) 



where the constant r^ is the same as in Assumption [3l without loss of generality, 
(d) There exists a real x* > such that 



VrQ{x) :-- 



r{x)Q'{x) - r'{x)Q{x) J > 0, for x < x*, if x* > 0, 



r(x) 



< 0, for X > X*, 



where the function Q is defined by (fTSj) . Also, if x* = 0, then \\va.xiQQ{,x) I "^{x) < oo. 
(e) The integrability condition 



E 



-a:(-") 



[\h{X^)\ + \CxK{X?)\]dt 



< oo 



is satisfied, and 

K{x) = -E 
for every initial condition x > 0. 



CxK{X^) dt 



(20) 
(21) 

n 



Remark 2 It is worth noting that fl2Tl) is in essence an integrabihty condition. Indeed, an 
application of Ito's formula yields 

e-^T''°^K{X^^) = K{x)+ [ e-^'^^''''^CxK{X^)dt+ [ e-^*'''°V(X°)A;(X°) t/I^t. (22) 











If we assume that the stochastic integral appearing in this identity is a martingale and that 
the so-called transversality condition 



lim inf E 

T-^oo 



e-^r 'k{X!^ 



T+J 







holds, then we can take expectations in (p2l) and then pass to the limit T ^ oo to obtain 
(ETD. D 



Remark 3 For future reference, we observe that the integrabihty condition (120]) implies 
that 



E 



\Q{X',)\dt 



< oo. 



Furthermore, if we define 



Rx^h{x) = E 



-,-A 



r(xO) 



h{Xf) dt 



then 



(23) 



Rxo,h{x)-K{x)=^ 



°° .^ri^') 



g(x°) dt 



uo 



=: Rxo,q{x 



thanks to ([2ID- 



(24) 
D 



The following result is concerned with the well-posedness of our optimisation problem. 

Lemma 1 Suppose that Assumptions\^ [H 0, \^(a), \^(h) and\^(d) hold true, and fix any 
initial condition x > 0. The performance index Jx{Z) is well-defined for every admissible 
intervention strategy Z E A, and v{x) < oo. 

Proof. Fix any initial condition x > and any admissible intervention process Z E A. 
Using Ito's formula, the fact that AX^ = AZt and the definition (TTB]) of the function K, we 



obtain 



e--^""K(XT+)^K(x)+ I £-'''•'" CxK(X,)dt+ / e'''''"" k(X,)dZi 



0<t<T 



-K 



-T 



(X) 



[K{Xt+) - K{Xt) - k{Xt)^Xt] + Mt 



K{x) + / e 
'o 



/ e"^* CxK{Xt)dt+ / e~^* k{Xt) dZ'^ 
Jo Jo 



+ E 



e-^; 



(X) 



0<t<T 



Xt- 



k{s) ds + Mt, 



Xt 



where Z'^ is the continuous part of the increasing process Z and 



Ml 



Kr(X) 



a{Xt)k{Xt) dWt. 



In view of ([9]) and the definition flTSl) of Q, this calculation implies that 

e-^^^'''' h{Xt) dt - [ e-^^^'''^k{Xt)odZt = K{x)-e-^T''^K{XT, 



+ 



T 



^r(X) 



Q{Xt) dt + Mt. (25) 



Now, combining the assumption that r{x) > tq > 0, for all a; > 0, with Assumption [51(d), 
we can see that the function Q/r is bounded from above. It follows that 



T 



e-^**"'Q(X,)dt<sup^ 



e-^^'"V(X,)dt = sup^<oo. 
>o r[s) Jq s>o r{s) 



(26) 



Also, the assumption that h/r is bounded from below implies that 
^^-^^'^'MX,)rft>inf^(' 



^*'''V(Xi) dt > inf ^ > -oo. 



>o r(s) 



^>o r{s) Jo 

In light of these inequalities, the fact that K is positive and real- valued, and (125|) . we can 
see that 

T 



inf Mt > inf ^ - sup ^ - K(x) 
t<T s>o r{s) s>o r(s) 



-A. 



r{X) 



A;(Xi) o dZt. 







The admissibility of Z implies that the right-hand side of this inequality belongs to C^, for 
all T > 0. It follows that M is a supermartingale. Combining this observation with (1261) . we 
can see that fl25|) implies that 



E 



r(X) 



h{Xt) dt 



Kr(X) 



k{Xt) o dZt 



'0 JO J 

However, these inequalities establish the claims made. 



< sup -^ + K(x) < cx). 
s>o r(s) 



D 
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3 The solution to a second order linear ODE 

In this section, we review some properties of the solution to the ODE 

Cxw{x) + G{x) = la^{x)w"{x) + b{x)w'{x) - r{x)w{x) + G{x) = 0, (27) 

that is associated with the Ito diffusion ([3]), and we estabhsh some new results that are 
concerned with appropriate analytic and probabilistic expressions for the derivative w' of a 
solution w to fl27j) . Indeed, if G is absolutely continuous, then differentiating the ODE (|27jl 
yields 

la^{x)w"'{x) + [b{x) + a{x)a'{x)] w"{x) - [r{x) - b'{x)] w'{x) - r'{x)w{x) + G'{x) = 0. 

Using fl27j) once again to eliminate w{x) from this equation, we can see that w' solves the 
ODE 

Cyu{x) + VrG{x) := la^{x)u"{x) + n{x)u'{x) - p{x)u{x) + VrG{x) = 0, (28) 

where /i is defined by flTTl) . p is defined by f flQl) . and 

It follows that w' satisfies a second order linear ODE that is similar to (!27j) and is associated 
with the SDE (^. 

In the presence of Assumptions [T|, [2] and [3|, the general solution to the homogeneous ODE 
Cxwi^x) = 0, which is associated with fl27|) . is given by 

w{x) = A(j){x) + Bip^x), 

for some constants A,B &W, where and ip are C^ functions such that 

< (f){x) and (f)'{x) < 0, for all a; > 0, (30) 

< ^{x) and tjj'ix) > 0, for all x > 0, (31) 

and 

lim0(x) = lim iIj{x) = oo. (32) 

These functions are unique, modulo multiplicative constants. To simplify the notation we 
assume, without loss of generality, that 

0(c) = V^(c) = 1, (33) 

10 



where c > is the same constant as the one that we used in the definition (j3]) of the scale 
function p^o. Also, these functions satisfy 

0(x)^'(x) - (t)'{x)^{x) = Cp'j,o{x), (34) 

where 

C:=[V'(c)-0'(c)]>O. (35) 

Furthermore, we can use the fact that and ip satisfy the ODE Cxwlx) = to verify that 

2t ( X] 
(j)" {x)%Ij' {x) — (p' {x)%ij" {x) = [(j){x)ip' (x) — (f)'{x)tp{x)] 

a'^{x) 

= ^-^pU^)- (36) 

cr^(a;) 

Similarly, Assumptions [H [3], H] and [51 (c) guarantee that the general solution to the ho- 
mogeneous ODE Cyu{x) = 0, which is associated with fl28l) . is given by 

u{x) = A(j){x) + Bip{x), 

for some constants A^B E^^ where and if) are C^ functions satisfying 

< ^{x) and 0'(s) < 0, for all a; > 0, (37) 

< i){x) and ^'{x) > 0, for all x > 0, (38) 

and 

lim0(x) = lim iIj{x) = oo, (39) 

Once again, we assume, without loss of generality, that 

0(c) = V^(c) = 1, (40) 

for notational simplicity. Also, we note that 

(p{x)lp'{x) - (j)' {x)i>{x) = Cpyoix), 

where 

C ■= ip'{c) - 0'(c) > 0. (41) 

The following example shows that Assumption [51(c) is indispensable if we want the 
functions and tp to have the properties f[371) - f[39|) . which are essential for constructing the 
solutions to many one-dimensional stochastic control problems, including the one that we 
study in this paper. 

11 



Example 1 Suppose that b{x) = 2x, a{x) = \plx and 



r\x] 



In this case, p{x) 



and Assumption [51 (c) is not satisfied. A simple calculation reveals that the functions 

3 1 

0(x) = X~2, Ipix) = X2 

span the solution space of the ODE CxU!{x) = 0, and that the functions 

4>{x) = x~2^ iIj[x) = X~2 

span the solution space of the ODE Cyu^x) = 0. Clearly, ijj does not satisfy (1551) and fl5^ . 

The next result expresses the functions and ip in terms of the functions and ip. 

Proposition 2 Suppose that Assumptions CI 11, and IB (c) hold. If 0, -0 and 0, -0 
are i/ie functions satisfying ^3U\)-^3^ and (37^-^^), and spanning the solution space of the 
homogeneous ODEs Cxw{x) = and Cyu{x) = associated with [2l\ ) and ^2^, respectively, 
then 



0'(c) ^'(c) 



(42) 



Also, both of and ip are convex, and, if C and C are the constants defined by l[3^) and 
[4^ > respectively, then 



c 



2r(c) 



O"" 



(c) 0'(c)^'(c 



■C. 



(43) 



Proof. We first show that ip = C^ip', for some constant C^ > 0. To this end, we define the 
function ip by 



■0(a;) 



0(s) ds. 



(44) 



Given an absolutely continuous function / :]0, oo[— ;> M with compact support, we can use 
the integration by parts formula to calculate 



f\s)Cx^{s)ds = - f{s 



r'is] 



Cyi,'{s) + -^Cx^{s) 
r[s) 



ds 



Since ip' = ip satisfies the ODE £y'u(x) = 0, it follows that 

r{s)ns)+r'{s)f{s) 



r(s) 



-£xV'(s) rfs = 0. 



(45) 



12 



Now, fix any a > and any z G ]0, a[, and define 

{1, if X G [a — z, a], 
-1, if X G]a, a + 2;], 
0, otherwise. 

If we choose 



r{x) Jo 
then / is absolutely continuous with compact support, and (H5]) yields 



. r{s) Ja r{s) 

Since z G]0,a[ has been arbitrary, we can differentiate this expression with respect to z to 
obtain 

^xf{a-z) ^ Cxf{a + z) ^ for all z g]0, a[. 
r[a — z) r{a + z) 

It follows that the function Cx'ip/r has even symmetry around the point a. However, since 
a > has been arbitrary, this can be true only if Cx'^P/r is constant. Therefore, there exists 

a constant Ci G M such that Cx ii^ix) + CA =0. If we combine this observation with the 

facts that 

i/j is strictly increasing, limiplx) G M and lim iIj{x) = cxd, 

which follow from the definition (jUj) of ip and the properties fl551) - fl5^ of the function ijj, 
and the properties (l30!l - (!32l) of the functions 0, ip that span the solution space of Cxw{x), 
we can conclude that ip + Ci = C^ip, for some constant C^ > 0. However, this conclusion 
establishes the identity ■?/' = C^iIj' . 

Now, since 0' and if)' = CT-^ip are independent solutions to the ODE Cyu^x) = and 

£y(/)(x) = 0, there exist constants C^ and F such that (p = —C^cj)' + T%1). However, the limits 
lim 4>{x) G [0, 00 [, lim 0'(a;) = and lim ifj^x) = 00 

X— >oo X— >oo X— >oo 

imply that F = 0, which proves that = —C^ip', for some constant C^ > 0. 
Finally, we note that 

-C^ = -—— and C^ 
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are the only choices for the constants C^ and C^ that are compatible with fj40|) . Also, fj43l) 
follows by a simple calculation involving the definitions of the constants C and C*, fj42l) and 

dMD. □ 

To proceed further, we consider the solution X^ to the SDE ([3]) and we define 



We recall that, if G is a measurable function, then 



if and only if 



E 



- |G(.)|^(.) 



\G{X^^)\dt 



< CXD 



(46) 



■ ds + 



|G(.)|0(.) 



(is < oo. 



In the presence of these equivalent integrability conditions, the function Rxo^g defined by 



(47) 



.r{X") 



G{X^) dt 



Rxo,g{x) = E 
admits the analytic expression 



(48) 



2 r 

Rx»,g{x) = — 0(x) / 



a^{s)p'^o{s) 



2 r 



(^^is)p'xois 



■ ds. 



(49) 



where C > is the constant defined by ( l35l) . and is a special solution to ( 1271) . 
At this point, we establish the following technical result that we will need. 



Lemma 3 Suppose that AssumptionsUWE cmd\Mhold. If G is a function satisfying ( f^^ and 
^1\ ), then 



|G(x)|^'(x) 



lim inf 



IG X 0' X 
lim inf ' ,\ , , ! = 0. 
x^oo r{x)p'^o[x) 



(50) 



Proof. In view of (1341) . we can see that 



0<^^^<1 and 0< 



(f)'{x)^{x) ^ ^ 
Cp'xoix) 
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which, combined with (l32l) imphes that 



Also, the calculation 



,i„,4^=,taf(£) =0. (51) 



d f 1 \ 2b{x) 



dx \p'xoix)J a'^{x)p'^oix) 
and the fact that (p satisfies the ODE Cxwlx) = 0, imply that 
d ( (j)\x) \ 2 



dx \p'xoix)J a'^{x)p'^o{x) 
2r{x)(j){x) 



[la^{x)<j)"{x) + b{x)<j)'{x)] 



a^{x)p'j^oix)' 
Similarly, we can show that 

d f ip'^x) \ 2r(a;)^(x) 



dx \p'xo{x) J o-'^{x)p'xo{x) 
Now, we consider any sequence (xn) such that 

o<,„<i a„d BM< i„f m + i. 

n r{Xn) xG]o,i[ r{x) 

Using (15T|) and (152|) . we calculate 



|G(x„)| ^^^ ^'(x„) 



lim„_,oo 
n — > oo in this inequality to obtain 



(52) 



(53) 



In view of f lSTj) and the fact that lim„_^oo /q^" a^(s)' (I) ^^ ~ '-'' *^ ""^^ P^^^ ^° ^^^ limit 



j.^ |G(x„)|^-(x„) ^ ^^ 

n^oo r{Xn)p'xoi.Xn) 

which proves that liminfa-io , f J fl = 0. Using similar arguments, we can also show that 
liminf M£M(£il = o D 

^'^'^ r{x)p'ix) ^- '-' 
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Remark 4 It is worth noting that the conclusions of the preceding result cannot be strength- 
ened. To see this, suppose that b{x) = —x, a{x) = \plx and r{x) = 3, so that 

(f){x) = 1/x, ip{x) = x^ and p'xo{x) = x. 

Also, let G be the positive function given by 

oo 

G{x) = J2g^''\x), 

n=l 

where C-"^ is the tent-like function defined, for n > 1 by 
G^'^Hx) - 



n^x + n — n^, 



In rf^ ' n L ' 

—n'^x + n + n^, for x G [-, - + t- T , 
0, otherwise. 



Given any x > 0, we calculate 

\G{s)ms 



ds< G{s)ds 
cr^{s)p'xo{s) Jo 

oo _. 



and we can immediately see that 



n=l 
< CX), 



\Gisms) 



1 /I 



n W \n n 



a 



'is)p'o{s) 



ds < oo. 



It follows that the assumptions of Lemma [3] are satisfied. However, 



IGjxMJx 
hmsup . w / N '- — 
xjo r{x)Pj^o{x) n-.oon 



> lim -G^") (-] > 0. 



n 



n 



In what follows, we assume that G is absolutely continuous. Also, we consider the solution 



r° to the SDE (ITOD and we define 



A^ "^ = / PiY^') dv, 
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If V^G is defined by fl29|) . then the standard theory discussed above Lemma [3] imphes that 



E 



,-Af 



\DrG{Y^)\dt 



uo 



< oo 



if and only if 



\VrG{s)\i,{s) 



ds + 



\V,G{sms) 



ds < oo. 



/o (r'^{s)p'yo{s) j^ a'^{s)p'yo{s) 

In light of fH2]) in Proposition [2] and fll2p . we can see that (1551) is true if and only if 

r{s)p^o{s) A 



(54) 



(55) 



(56) 



r{s)p'xois) 
Furthermore, if these equivalent conditions hold true, then the function Ryo.VrG defined by 



R 



Y°,VrG 



x) =E 



°° _AP(y°) 



VrG{Yl^) dt 



admits the analytic expressions 



Ry.,vM-)-l4>{^,j^^ .^(.K,(.) 






{VrG){s)^{s) 
^^(s)Pyo(s) 



(is 



1 ■ ^'„,) /^ (^-g)wv/(.^) ,^ ^ 1 ^,,,^, r (p.G)M[-^i(.) ^^_ 1^^, 



G 



'^(s)pVo(s) 



C 



'"(s)Pxo(^) 



where G and (7 are defined by (l35l) and PTI) . respectively, and provides a solution to the 
ODE Cyu{x) +T>^G{x) = 0. Once again, the second equality here follows from Proposition [2] 
and ([12]). 



Proposition 4 Suppose that Assumptions [21 '^^^ S ('^) hold true, and let G he a 
function satisfying (^^ and (2^- Also, suppose that there exists a constant e > such that 



and 



either VrG{s) < 0, for all s < e, or V^G^s) > 0, for all s < e 



either VrG(s) < 0, for all s > l/e, or VrG{s) > 0, for all s > 1/e. 



Under these conditions, the integrability condition ^5^ holds true, the function Ryo^VrG is 
well-defined and real-valued, and 

R'xo ci^) = RYO,VrG{x), for all X > 0. 
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Proof. In view of Lemma [3l fl52|) . fl53|) and the relationship fl29l) of the functions G and 
Vj-G, we can use the integration by parts formula to obtain 

{VrG){sW{s) ^^ ^ G{x) fix) _ 2 r G^)^^) ^^ 



h r{s)p'^ois) r{x) p'^oix) Jq a^{s)p'^ois) 

and 

{VrG){s) [-0'] is) ^^ ^ Gjx) <P'{x) ^ 2 r Gis)<p{s) ^^^ 



r{s)p'xo{s) r{x) p'^o{x) J^ a^{s)p'^„{s) 

However, combining these expressions and the expressions fH9|l and (l57j) for Rx<-\g and 
-Ryo,x>rG with the assumptions on V^G and its local integrability we can see that all of the 
statements made hold true. D 



Remark 5 If we remove the assumption that VrG{x) has constant sign for all x sufficiently 
small and for all x sufficiently large, then the conclusions of the result above, ( I55l) in partic- 
ular, do not necessarily hold. To see this, suppose that b, a and r are as in Remark HI and 
let G be the function given by 



G{x) = J2g^ 



")^x) 



n=l 



where C-"^ is the tent-like function defined by 

(2n{2n + l)x-2n, ioi xe[^,^], 
G(")(x) = <^ -2n{2n - l)a; + 2n, for x e [^, ^] , 
[O, otherwise. 

Plainly, G satisfies (1461) and fH7|) because it is bounded. However, the calculation 



r(sW,.M ^^-i^\^'Gis)lds 



2_^2n{2n + l) s ds + 2_^2n{2n - 1) s, 

n=l 2ri+l n=l 277 

E4n + 1 ^^ 4n - 1 

4n(2n -I- 1) ^ 4n(2n - 1) 

ra=l ^ ' n=l ^ ' 



CX) 



shows that ( !56|) is not satisfied. D 



The optimal strategy of the control problem that we solve in the next section reflects the 
state process in a given point a > in the positive direction. Such a strategy involves the 
construction of a continuous increasing process Z°- such that, if X°- is the associated solution 
to the SDE (ED with x> a, then 



XI > a and Z^ = / l{x-=a} dZ^, for all t > 0. 

Jo 



(5J 



Such a construction is standard and can be found in El Karoui and Chaleyat-Maurel |EC78] 
(see also Schmidt |Sc89j ). 

Lemma 5 Suppose that AssumptionsUl 0> (^ndl^fc) hold. Given a real number a > 0, 
consider the continuous increasing process Z"" and the solution X"' to the SDE ^ with initial 
condition x > a that satisfy (E^j- // G : ]0, oo[ — ;> M is a measurable function satisfying [J^ ) 
and [4^ , then 



E 



e-^* \G{X^)\dt 



< oo, 



(59) 



and 



Ux'^,g{x) :-- 



-^xo,g(") 



(x) + Rxon{x) 



E 



e"^* G{X^)dt 



where Rxo,g is defined by ( f^Py . Furthermore, 



lim E 

n— >oo 



R 



XO,G 



'n) = 0, 



where T^ is the first hitting time of {n} defined by 

T^ = mi{t > I Xf = n}. 



(60) 



(61) 



(62) 



Proof. In view of the continuity of Z"- and the fact that it increases on the set {X^ = a} 
(see (EH])), we can see that Ito's formula and the identities CxUx<^,G(yX) + G{x) = and 
U'xa fj(a) = imply that 



e-^^'""Vx^G(X^) = f/x^G(a;) + f e-^*'""'/:xf/x^G(Xf 

Jo 



dt 



Ux^,g{x) - I e-^*'" 'G{Xt) dt + M^, 



(63) 
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where 



M° 



r{X") 



e-^* aiX^)U'^.^^iX^)dWt. 



Ito's isometry and the continuity of a and U'xa q imply that 



E 



(Mt,)' 



E 



L{t<T„n 







,r(X") 



e-^* a(Xnf/i.G(^n 



dt 



< 7^ sup [fT(s)f/^a_G(s)]' 
-^^0 se[a,n] 

< oo, 



where ro is as in Assumption [3l It follows that the stopped process (M °) " is a uniformly 



square integrable martingale, and therefore, E 
observation and fl63|l . we obtain 



(M' 



oo 



E 



MS^a 



0. In view of this 



E 



e -"" 






Ux^,Gin) = Ux^,Gix) - E 



e"^' G(X,'') dt 



Uo 



(64) 



Now, if G is bounded, then fl60|) follows immediately by passing to the limit n ^ oo 
in fl6^ using the dominated convergence theorem, the fact that the restriction of Ux°-,g in 
[a, cxd[ is bounded and the assumption that r(x) > tq > 0, for all x > 0. If G is positive, we 
define G'^'") = G Am, for m > 1, and we note that 



U Xa n'-m'l \X) 






(x) + R 



XO,G(™) 



:x) = E 



;-A, GM(Xf)t/t 



(65) 



(is, 



because G^'") is bounded. In view of (1491) . the calculation 

i?xo,c(-)-^0(-)y^ ^^(.)i>:,o(.)^'+G^^"V. a^(.)p:,o(.) 

and the monotone convergence theorem, we can see that 

lim RxQ G(n^){x) = Rx(\g{x) and lim R'oc(m){a) = R'xOGi'^) 



because G satisfies ( 1471) . and that 



lim E 

m^oo 



,r(X") 



g(")(^")^^ 



E 



e-^' G{X^)dt 



However, these limits and fl65|) imply (l60l) . The general case now follows by considering the 
minimal decomposition G = G^ — G^ of the function G to the difference of two positive 
functions, and by linearity. 
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To complete the proof, we note that, if G satisfies P7|) . then Ux'^,\g\{x) < oo, which, 

59|) . Also, flM|) . fl59l) . fl60l) and the dominated convergence 



combined with fl60l) . implies 
theorem imply that lim„_^oo E 



f/xa,G(^) = 0. However, this limit, the fact that the 



restriction of in [a, oo[ is bounded and Assumption [ni(c) imply (IMl) . D 

4 The solution to the control problem 

With regard to standard theory of stochastic control (e.g., see Chapter VIII in Fleming and 
Soner |FS93j ). we expect that the value function v identifies with an appropriate solution w 
to the Hamilton- Jacob i-Bellman (HJB) equation 

m.six{CxU!{x) + h{x), w'{x) — k{x)} = 0, (66) 

where Cx is the operator defined by (IT7|) . We conjecture that the optimal strategy of our 
problem can take one of two qualitatively different forms, depending on the problem data. 
The first of these arises when it is never optimal to exert any control effort. In this case, we 
expect that the function Rxo^h that is defined by fl23l) in Remark [3] should satisfy fl66l) . 

The second one is characterised by a boundary point a > and can be described as 
follows. If the system's initial condition x is less than a, then maximal control should be 
exercised to immediately reposition the system's state at level a (i.e., cause a "jump" of size 
a — a; in the positive direction at time 0). After this possible initial jump, minimal control 
should be exercised so that the system's state process is reflected at the boundary point a in 
the positive direction. In view of the heuristic arguments that explain the structure of the 
HJB equation (!66|) . if this strategy is indeed optimal, then we should look for a function w 
and a point a > such that 

CxU!{x) + h{x) = 0, for x > a, (67) 

and 



wix] = w[a 



r 

— / k{s) ds, for X < a. (68) 



Now, every solution to the ODE in (1671) is given by 

w{x) = A(j){x) + BiIj{x) + Rxo,h{x), (69) 

for some constants A,BeW, where the functions (p and tp are defined as in Section [31 It 
turns out that the arguments that we use to establish Theorem [7] below, which is our main 
result, remain valid only for the choice B = 0. For this reason, we look for a solution to the 
HJB equation fIBB]) of the form 

_ \A(t){x) + Rxo,h{x), for x>a, 
1 w{a) — jP^ k(s) ds, for x < a. 
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To specify the parameter A and the free-boundary point a, we appeal to the so-called 
"smooth pasting" condition of singular stochastic control that requires that the value func- 
tion should be C^, in particular, at the free-boundary point a. This requirement gives rise 
to the system of equations 

R'xo,hi(^) + A(j)'{a) = k{a), 
R';,o^hi(^) + A(P"{a) = k\a), 

which is equivalent to 

k{a) - R'xo,hi(^) _ k'ja) - Rxo,hi(^) 

0'(a) ~ 0"(a) ' ^ ' 

0'(a) [i?:^o,,(a) - k'ia)] - 0"(a) [R'^o^^ia] - A:(a)] = 0. (72) 

Taking note of (!24|) in Remark [3l we can see that 

R'xo,,ix)-kix) = R'xo,Qix), (73) 

so (1721) is equivalent to 

0'(a)i?:^o,Q(a) - 0"(a)i?:^o,Q(a) = 0. (74) 

Now, Proposition H] and (1571) imply that 

where the function V^Q is as in Assumption [51(d). Combining this observation with the 
calculation 



we can see that ( 1741) is equivalent to 



[0"(a)^'(a) - ^'{aW\a)] / ^^f.^^.'^f ^'^ rf. = 0. (77) 



In view of the fact that (f)"il)' — (p'lf)" is strictly positive (see also fl36|) ). we conclude that fl72!) 
is equivalent to 

The following result is concerned with the considerations above regarding the solvability 
of the HJB equation fl66l) . 
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Lemma 6 Suppose that Assumptions\M^CLre satisfied. The equation g{a) = has a unique 
solution a > if and only if 

X* > and limg{x) < 0, (79) 

xj.0 

where x* > is as in Assumptionl5[ (d) . Furthermore, the following cases hold true: 

(I) If ( fy^ is not true, then the function Rxo,h satisfies the HJB equation (E^)- 

(II) If ( fyPl J is true, a > is the unique solution to the equation g{a) = and A > is 
the constant given by [71^ , then the function w defined by ([7^ is C^ and solves the HJB 
equation (E^. 

In either case, the associated solution w to the HJB equation (E^) is bounded from below. 
Proof. In view of the definition fl78|) oi g and Assumption [51(d), we calculate 

iVrQ){x)(P'ix) /> 0, for X < X*, iix* > 0, 
r[x)p^o{x) \^<0 ioTX>x*. 

Combining this calculation with the fact that lim^j^oo g{x) = 0, we can see that the equation 
g{a) = has a unique solution a > if and only if (1791) is true. For future reference, we also 
note that 

if there exists a > such that g{a) = 0, then a < x* and g{x) > 0, for all x > a. (81) 

In this case, ( IHOl) and the fact that (p' < < tp' imply that 

-— - — 1— — > 0, tor all X < a, 

r{x)p^o{x) 

which, combined with (17T1) . (1731) . fl75l) and (1781) . implies that 

(f)'{a) C Jo r{s)p'^o{s) 

On the other hand, 

if the equation g{a) = has no solution a > 0, then g{x) > 0, for all x > 0. (82) 

With regard to Case I we will prove that Rxo,h satisfies the HJB equation fl66l) if we show 
that R'j^a i^{x) — k{x) < 0, for all x > 0, which is equivalent to showing that 

RYO,VrQ{x) < 0, for all x > 0, (83) 
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thanks to fl73l) and fl75l) . To this end, we use fl75|) and fl76|) to calculate 
d fRY^,VrQ{x)\ (t)"{x)ij'{x)-(t)'{x)ij"{x) 



dx 



-0'(X) 



C[<P'{x)y 



-9{x). 



(84) 



The right-hand side of this identity is strictly negative for all x > 0, thanks to the strict 
positivity of the function 0'V' ~ 0V (see (1361) ) and ( l82l) . Also, in view of ( l82l) and the fact 
that 0' < <■?/'', we can see that 



lim^i;i:^ = ilim 



(^'«(^'>*'(4,, + ^<,(,) 



r{s)p'^o{s) 



(f)'{x) 



<0. 



However, these observations imply fl83l) . 

By construction. Case II will follow if we show that 



(7'^{x)k'{x) + b{x)k{x) — r{x) 



1,2 
2 



w a 



A;(s) ds 



+ h{x) < 0, for all x < a, (85) 



w'(x) — A;(a;) < 0, for all x > a. (86) 



In view of the definitions flT6|) and (IT8|) of the function K and Q, respectively, we can see 
that fl85|) is equivalent to 



(5(2;) + r{x)K{a) — r(a;)t/;(a) < 0, for all x < a. 
Also, the facts that w is C^ and satisfies fl671) imply that 



r{a)w{a) 



y'[ 



a)k'{a) + b{a)k{a) + /i(a) 



= (5(a) +r{a)K{a). 
These observations and the strict positivity of r imply that (!85|) is equivalent to 

Q{x) ^ Q{a) 



< 



for all X < a. 



r(x} 



r(a) 



However, this inequality follows immediately from the fact that ( ISTI) and Assumption [51(d) 
imply that (Q/r) (x) > 0, for all x < a. 

Substituting the first expression in ( ITTI) for A in ( ITOl) . and using ( 1731) and ( 1751) . we can 
see that (l86l) is equivalent to 

RYO,VrQ{a) . ^yo,©,Q(a;) 

-TT^ - ,// N > fo'^ all a; > a. 
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However, this inequality follows immediately once we combine (18^ with the strict positivity 

of 0'>' - 0V" and dHU) to obtain 

d (Ry^vA^),^^^ for all a: > a. 



dx \ —(j)'{x) 

Finally, we note that the assumption that h/r is bounded from below (see Assump- 
tion [51(a)) and ( l23l) in Remark [3] imply that Rxo,h is bounded from below. However, this 
observation and the structure of the solution w to the HJB equation ( l66l) associated with 
either of the two cases considered imply that w is bounded from below. D 

We can now prove the main result of the paper. 

Theorem 7 Suppose that AssumptionsU^{^ hold. The value function v of our control prob- 
lem identifies with the solution w to the HJB equation (ESP derived in Lemma O In partic- 
ular, the following two cases hold true: 

(I) If the problem data are such that (79) is false, then it is optimal to exert no control effort 



at all times. 

(II) If the problem data are such that (79) is true, then the optimal intervention strategy 



involves a jump of size {a — x)^ at time and then reflects the state process X in the 
boundary point a > in the positive direction. 

Proof. Fix any initial condition a; > and any admissible intervention strategy Z E A. 
Using Ito's formula and the fact that AXt = AZf, we calculate 



e T 



^^""w(Xr+) = w(x) + /" e-^^^^''^Cxw{Xt)dt+ f e-^^^^""^ w'{Xt) dZt 

Jo Jo 



0<t<T 



pi pi 

w{x)+ e-^^^^""^ Cxw{Xt) dt + e-^*''"w'(Xi)dZ 
Jo Jo 

+ Yl e-''^^'''\w{Xt + AZt)-w{Xt)] + MT, 



0<t<T 

where the operator Cx is defined by (fTTIl . and 



Mt= e-^'"'V(Xi)w'(Xi) dW^i. 
Jo 
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In view of this calculation, IQ and the fact that w satisfies the HJB equation fl66l) . we obtain 



/•T rT 

/ e-^^ h{Xt) dt - e'^^ k{Xt) o dZt 
Jo Jo 



w{x) -e ^T 



r(X) 



w{X. 



-A 



r(X) 



T+j^ I e ■- [/:xw^(Xi) + h{Xt)\ dt 





+ 1 e 

'o 



r(X) 



-''^'"[w'{Xt)-k{Xt)]dZ^ 

-AZt 






0<t<T 



^r(X) 



<w{x)- e-^T w{Xt+) + Mt. 



n) 



To proceed further, let (r„) be a localising sequence for the stochastic integral M. Taking 
expectations in (187|) . we obtain 



E 



T,iAT pTn/\T 

e"^* h{Xt)dt- 
Jo 



e-^*'"'A;(X,)orfZ, 



< w;(a;) + E 



^--i^) 



(88) 



where w~ = — (w A 0). Now, the assumption that h/r is bounded from below, the fact that 
the process B defined by Gj = — exp ( — A[ j , for t > 0, is increasing, and Fatou's lemma 
imply that 



E 



r(X) 



h{Xt) dt 


= E 


\fVM 

Jo r{Xt) 




< hn 

n- 


linfE 

->oo 


Jo 



^^^').e. 



r{Xt 

liminfE / e'^^ h{Xt)dt 

"^°° iJo 



"TnhT 



(89) 



while the monotone convergence theorem implies that 



E 



Kr(.X) 



k{Xt) o dZt 



lim E 

n— »oo 



T„AT 



r{X) 



e-^* k{Xt) o dZt 



Also, the fact that w is bounded (see Lemma [6]) and the dominated convergence theorem 
imply that 



lim E 

n^oo 



r{X) 



-nAT^i; (X(^^^y)^) 



E 



-^-"'./;-(Xt+)' 



26 



In view of these observations, we can pass to the hmit n — > cxd in flHHj) to obtain 



E 



Uo 



e"^* h{Xt)dt- / e-^' k{Xt)odZt 
Jo 



r{X) 



<w{x)+E e'^T w-{Xt+) 



Combining this inequahty with the hmit 



hm E 



-a;(^) 



e-'^T w-{Xt+) 



0, 



which foUows from Assumption O (c) and the fact that w is bounded, we can see that 



Jx{Z) = hmsupE 



T^oo 



-,-A 



r(X) 



h{Xt) dt - 



r(X) 



k{Xt) o dZt 



< w{x). 



(90) 



If ( !79|) is false, then the control strategy Z° = has payoff 

Jx{Z°) = Rxo^hix) = w{x). 

The first equality here follows from the definition (!23l) of Rxo^h and the dominated conver- 
gence theorem (see (120|) in Assumption [51(e)), while the second one is just Case I of Lemma [61 
However, these identities and (1901) establish Case I of the theorem. 

Now, let us assume that (17^ is true. Let Z°- be the caglad process that has a jump of 
size (a — x)"*" at time and then reflects the state process in the boundary point a > in the 
positive direction (see the discussion preceding Lemma [5]). Also, let X"- be the associated 
solution to the SDE ([3|) and let T^ be the first hitting time of {n}, which is given by f[62l) in 
Lemma [5l In this case, we can check that ( IHTl) holds with equality, so 



-K 



{X°) 



h{X^) dt - 






k{X^) o dZ^ = w{x) - e-^T w{X^)+M^, (91) 



for all T > 0, where 



M^ 



rCX") 



a{X'l)w\X^) dWt. 



Using Ito's isometry and the fact that a and w' are continuous, we can see that, for n > aVx, 



E 



{MT,y 



E 



-{t<Tit} 



riX") 



^*^ a{X^)w'{X^) 



1 2 



dt 



1 ^ 2 

< sup [(7{s)w'{s)] 



2r, 



sG[a,n] 



< CO, 
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where tq is as in Assumption [31 It follows that the stopped process (M ") " is a uniformly 
square integrable martingale, so E (M'^)j" = E 
imply that 



E 



n-<a 



e'^^ h{X^)dt 



'-pa 



e-^''""'M^n°rf^f 



M^a = 0. This observation and (19T1) 

w(x]-E e~^^T^ w{n), (92) 



'0 ^0 

for all n > a V X. 

Combining the fact that w is the sum of Rx»,h and a bounded function with Assump- 
tion [51(c) and f[6T]) in Lemma [5l we can see that 



lim E 

n—*oo 



,r(X") 



W7(n) = 0. 



Also, Lemma [3 and the dominated convergence theorem imply that 



lim E 

n— ►oo 



,r(X") 



/i(Xf ) dt 



E 



,r(X") 



/i(X,") rft 



(93) 



while the monotone convergence theorem implies that 



hm E 



e-^5;'""'A:(XnorfZf 



uo 



E 



hr[X"-) 



These limits and f lU2]) imply that 



E 



,r(X") 



/i(Xf ) rft 



-E 



uo 



,r(X") 



fc(xn o dzf 



w UC 



If we combine this conclusion with f[93|) . then we can see that 



E 



A;(Xn o dZ^ 



IJo 



< oo, 



so Z"' is admissible. Furthermore, if we combine it with (1901) . then we can see that v{x) = 
w{x) and that Z°- is optimal. D 

5 Special cases 

We now consider special cases that arise when the running payoff function his a. power utility 
function and the running cost function k as well as the discounting factor r are constant. In 
particular, we assume that 

h{x) = Xx'^ , k{x) = K, and r{x) = ri, 

for some constants k. A, ri > and u G ]0, 1[. Also, we assume that the uncontrolled system's 
state dynamics are modelled by a geometric Brownian motion (Section 15. ip or by a mean- 
reverting square-root process such as the one in the Cox-IngersoU-Ross interest rate model 
(Section [ESD- 
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5.1 Geometric Brownian motion 

Suppose that X^ is a geometric Brownian motion, so that 

dX^ = bX^ dt + aX° dWt, X° = X > 0, 

for some constants b and a ^ 0, and assume that ri > b. In this case, it is a standard 
exercise to verify that, if we choose c = 1, then 

0(x) = x™, ^{x) = x" and p'xo{x) = x"+"^-\ 

where the constants m < < n are the solution to the quadratic equation 

yp +{b- ia^) / - n = 0. 

Also, it is well-known that 

ri> b ^ n> 1. (94) 

In this context. Assumptions [1], [2], [3], H] and [5l(a)-[5l(c) are plainly satisfied. Also, in the 
presence of ( 1M|) . we can calculate 



E 



e-'''^Xf dt 



X 



ri 



However, using this observation and the facts that 

< h{x) < A(l + x) and CxK{x) = — (ri — b)Kx, 

we can verify that Assumption [51(e) is satisfied. Furthermore, we can use the calculation 

VrQix) = Q\x) = Xux-^^-"^ - K{ri - b) 

to verify that x* > and that Assumption [5l (d) is satisfied as well. 

In view of the fact that n > 1, we can check that the function g defined by ( 1781) admits 
the expression 

g(x) = x"-"" ^— ^x^"", for X > 0. 

n — u n — \ 

It follows that Case II of Theorem [71 is always true and that the unique solution a > to 
the equation g[a) = {] that characterises the optimal strategy is given by 



Xuin 



K(ri — b){n — v) 



1/(1-1.) 
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5.2 Mean-reverting square-root process 

Suppose that X^ is a mean-reverting square-root process, so that 

dX^ = a{e-X^)dt + (T^/xfdWt, X° = x > 0, 
for some constants a,9,a > 0, and assume that 

ad- \o'^ > 0, 



(95) 



which is a necessary and sufficient condition for X° to be non-explosive. In this case, we can 
check that the associated diffusion y° satisfies the SDE 



dY^^ = a 



21 



e + 



a 
2^ 



Y^ dt + a vy; dWt, Y^ = x> 0, 



and that F" is non-explosive because a^ > 0, which is plainly true, is the condition corre- 
sponding to (jHSD- We can therefore see that Assumptions [T], [21 El ll]and[Sl(a)-[Sl(c) are all 
satisfied. 

Now, we calculate 



E 



-"^*Xj dt 



-r\t 



[6 + 



-"*] dt 



aO + Tix 



ri(a + ri) 

Combining these identities with the inequalities 

< h{x) < A(l + x) and \CxK{x)\ = \a6K, — K,{a + ri)x\ < aOn + K,{a + ri)x, 

we can verify that Assumption [51(e) is satisfied. Also, in view of the calculation 

VrQ{x) = Q'{x) = Xux-^^-''^ - (a + nn), 

we can see that x* > and that Assumption [51 (d) holds. 

Making the transformations y = laxja'^ and w{y) = w{x) in the ODE Cxw{x) = 0, we 
obtain 



w"{y) + 



2a9 



a^ 



ri 



y w'{y) w{y) = 



a 



which is Kummer's equation. With reference to Abramowitz and Stegun |AS72l Chapter 13] 
or Magnus, Oberhettinger and Soni |MOS66| Chapter VI], it follows that, if we choose c = 1, 
then 

f/(ii,2^;2a ) ^irjn,2^;2. ) 



TT {ri 2a6». 2a\ 



IP (ri 2ae . 2a\ 
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and 



p^„(x)=x-2"^/-'e2"(^-l)/'^^ 



Note that iFi is the well-known confluent hypergeometric function. 
Now, in view of the differentiation formula 

U'{a, b; x) = -aU{a + 1, 6 + 1; x) 

(see Abramowitz and Stegun |AS72^ 13.4.21]), we can see that the function g defined by ( 1751) 
is given by 



9{x) = 



\ a ' a'' ' a-' / 



oo 

2ae 






X 



- (a + Kri) / s'^ e '^ U [ hi, — ^ + 1; -^s as 

Jx V" ^ ^ / . 

This expression is complex, so we have to resort to numerical techniques to determine the 
solution of the equation g{a) =0. 
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